feat: add DM (Data Migration) support#6883
Draft
tennix wants to merge 6 commits into
Draft
Conversation
Introduces DMGroup/DM (dm-master) and DMWorkerGroup/DMWorker (dm-worker) CRDs following the existing Group+Instance pattern used by TiCDC, TiDB, and TiKV. Adds ComponentDMMaster/ComponentDMWorker constants to the meta package and regenerates deepcopy/register files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…worker - Register DM and DMWorker in InstanceSet, DMGroup and DMWorkerGroup in GroupSet - Add full Instance/Group interface implementations for DM master (DM, DMGroup) - Add full Instance/Group interface implementations for DM worker (DMWorker, DMWorkerGroup) - Add scope structs for DM and DMWorker with GVK, component, list helpers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add VolumeMountType constants and default paths for DM master (data) and DM worker (relay-dir) - Add container names, config dir paths, and cluster TLS dir paths to names.go - Add DM/DMWorker port utility functions to coreutil - Register DM container names in allMainContainers - Add dm config package (master-addr, peer-urls, initial-cluster, join, TLS) - Add dmworker config package (worker-addr, join, relay-dir, TLS) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add DMState, DMGroupState, DMSliceState, DMWorkerState interfaces to common - Add DM image constant (pingcap/dm) to image package - Implement dm-master instance controller with full task pipeline: - state: peer DM slice for initial-cluster computation - ctx: HTTP health check against /api/v1/cluster/info - cm: config map with initial-cluster or join address - pvc: DataVolume PVC + additional volumes - pod: dm-master container with ports 8261/8291, TLS support - status: sync MemberID and conditions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ses 5-8) - Add DMGroup/DMWorker controllers with Group+Instance pattern - DMGroup controller: headless+internal services, bootstrap annotation management, rolling update with maxSurge=0/maxUnavailable=1 - DM instance controller: configmap, PVC (DataVolume), pod, health check via /api/v1/cluster/info, status sync - DMWorkerGroup controller: headless service only, no bootstrap sequence - DMWorker instance controller: configmap, PVC (RelayVolume), pod, health check via /status, dm-master addr from DMGroupRef internal service - Register all four controllers in main.go with field indexes and cache Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add basic cluster examples for a 1-replica dm-master cluster (DMGroup) and a 1-replica dm-worker (DMWorkerGroup) pointing to it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
|
Skipping CI for Draft Pull Request. |
Contributor
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #6883 +/- ##
==========================================
- Coverage 37.44% 35.16% -2.28%
==========================================
Files 392 427 +35
Lines 22432 24054 +1622
==========================================
+ Hits 8399 8458 +59
- Misses 14033 15596 +1563
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
DMGroup,DM,DMWorkerGroup,DMWorkerwith TLS support from the startpkg/configs/dm,pkg/configs/dmworker) for dm-master and dm-workerGET /api/v1/cluster/info), ConfigMap, DataVolume PVC, pod lifecyclemaxSurge=0/maxUnavailable=1for pod name stabilityGET /status), ConfigMap, RelayVolume PVC, pod lifecyclecmd/tidb-operator/main.goexamples/basic/07-dm.yamlandexamples/basic/08-dmworker.yamlArchitecture notes
AnnoKeyInitialClusterNumannotation on instances to computeinitial-cluster;TaskBootremoves the annotation once all initial instances are ready. Bootstrap state is derived from existing instances (absence of annotation = bootstrapped).joinaddress is derived from<DMGroupRef.Name>-dm-masterinternal service at the default DM port (8261).maxSurge=0, maxUnavailable=1to preserve pod names required by dm-master'slastBoundetcd history.Test plan
examples/basic/07-dm.yaml— verify DMGroup creates headless + internal service and DM podsAnnoKeyInitialClusterNumannotation removed after all initial replicas readyexamples/basic/08-dmworker.yaml— verify DMWorkerGroup creates headless service and DMWorker pods<dmgroup>-dm-masterservicespec.tlsClusteron Cluster, confirm TLS mounts appear in pods🤖 Generated with Claude Code